Natural language generation (“NLG”) is a computer implemented processing task of generating natural language text. NLG can be thought of as a translator that converts a computer based representation into a natural language representation. There are various considerations to incorporate when trying to make computer generated text sound more “natural” such as what type of text is sought to be generated (communicative goal), what entities, events and relationships will express the content of that text, and how to forge grammatical constructions with the content into “natural” sounding text. These considerations are articulated in a variety of architectures. For example, Bateman and Zock (2003) summarize NLG as follows: (1) Macro Planning: determining how to structure a document (e.g., a document plan); (2) Micro Planning: determining how to phrase sentences within that document given a set of content (sentence and content planning); (3) Surface Realization: concatenating the information from (1-2) into coherent and grammatical text; and (4) Physical Presentation: document layout considerations (formatting, titles, etc.) (see J. Bateman and M. Zock (2003), “Natural Language Generation,” In R. Mitkov (Ed.), Oxford Handbook of Computational Linguistics, Research in Computational Semantics, pp. 284-304. Oxford University Press, Oxford.). Each one of these stages can have several subtasks and vary considerably in terms of complexity.
There are several known NLG systems: rule-based, statistical and template-based. Known rule-based NLG systems utilize domain dependent rules (e.g., the weather domain) and manipulate different stores of data to generate “natural” sounding text. The rules exist at all levels of the known system from selecting content, to choosing a grammatical output to post-processing constraints to, for example, aggregating sentences, resolving pronouns and varying content. For example, if a rule-based system were created to generate weather forecasts from numerical data, the numerical data would need to be analyzed. In addition, decisions about what content to convey based on the analysis and how to grammatically represent the content at the sentence and document level are made and implemented by hand-crafted, domain dependent rules with input from multiple experts. While these known systems do not need an existing domain corpus to create the rules, the rules are heavily dependent on a given domain and require linguistic and domain expertise to develop the rules. In other words, a criticism of rule-based systems is that despite the generated texts tending to be of an objective high quality, there is intensive reliance on human investment.
Another known system is a statistical NLG system. Known statistical NLG systems look to bypass extensive rule construction by using corpus data to “learn” the set of rules. A known statistical NLG system creates alternative generations of natural language text from the statistical rules and then chooses which alternative at a given point in a generated discourse is governed by a decision model. For example, the pCRU system, which generates weather texts, starts with a small number of relations that are trained on a corpus. Alternatives are generated and then different probabilistic methods are used to generate the natural language text (see A. Belz (2007) “Probabilistic Generation of Weather Forecast Texts” In Proceedings of Human Language Technologies 2007: The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT'07), pp. 164-171). While known statistical NLG systems are less domain dependent and reduce the need for domain expertise, they do need training data and there is no guarantee that the natural language text is grammatically correct. Statistical systems have less of a reliance on human investment, but can be computationally expensive, depending on the type of decision model in place, and the output texts can be shorter and less natural.
A third known system is a template-based NLG system. A known template-based system creates a template where empty slots are replaced by specific information. For example, a template sentence may be “The temperature is <value>.” The <value> is the empty slot where specific temperature information may be replaced. For known rule-based and statistical approaches, templates can be employed to structure the syntactic output. The use of templates avoids the need for extensive grammatical correction. The drawback is that templates from a known template-based system tend to be less interesting, less variable and less natural.
A central consideration in NLG is a concern for the adaptability of a given system to new domains. Significant human investments often need to be made to create systems from scratch. While these known systems may perform very well for a given domain, extending to alternative domains may require starting over. Statistical approaches, while streamlining some initial in-vestment in human resources potentially falls prey to the same concern. A goal is to find an appropriate balance between how much pre-processing of data is required and how many rules are necessary relative to achieving an appropriate level of evaluated acceptance of the generated text.
Accordingly, the inventors have recognized the necessity for additional improvements in natural language generation.